A Load Balancing Strategy for Iterated Parallel Loop Scheduling

An eecient template for the implementation on distributed-memory multiprocessors of iterated parallel loops, i.e. parallel loops nested in a sequential loop, is presented. The template is explicitly designed to smooth unbalanced processor workloads deriving from loops whose iterations are characterized by highly varying execution times. Experiments conducted shows performance gains w.r.t. HPF-like language supports. 1 Introduction The eeective exploitation of parallel loops is the key to achieve considerable speedups in many scientiic compute-intensive applications. Data parallel languages such as HPF rely on user-provided directives to derive the distribution of data and on the owner computes rule 4] to determine static schedules of parallel loops from the provided data distribution. The choice of the best data distribution (blocking or cyclic) is currently up to programmers. Note that, for some non{uniform problems, it is impossible to nd a good distribution scheme if both a static data distribution and a static scheduling policy are adopted. In fact, one may need a blocking data distribution strategy to exploit locality, but also a cyclic data distribution to balance the workload among the processors. Our template 3] overcomes this strong limitation of data parallel languages. It relies upon a simple blocking data distribution to exploit locality, and combines static and dynamic policies to obtain well-balanced iteration schedules.